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(54) METHOD AND DEVICE FOR LEADING INTERACTIVE LANGUAGE LEARNING 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide method and device for 
leading interactive language learning. 

SOLUTION: In the method and device for leading the interactive 
language learning, a text file for processing is displayed, the basic 
functions for interactive learning are provided, an animation of a 
face is displayed, and a work space for a language construction 
function is provided. This system is provided with a set of 
language rules stored as part of a text/voice conversion sub- 
system, and another set of stored rules applied to a process for 
learning the languages. The method realized by this system 
includes a step for converting the text to an audible voice, a step 
for supplying the audible voice to a user or a student (together 
with an animation image when selected), a step for prompting the 
student so as to repeat the audible voice, a step for comparing the 
repeated voice of the student with the audible voice provided by 
the system and a step for providing a student with feedback and 
support, e.g. for selecting a required part and reproducing the audible voice and the voice of the 
student toward the student. 
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Detailed Descriptions of the Invention: 



[0061] 

Feedback on performance may be given while the student is reading the text or after task 
completion. While the student is reading the text, the Verbal Information Verification processing (or 
utterance verification processing) can be used to display real-time performance feedback. The 
system may use any number of graphical or audio cues to indicate performance including but not 
limited to bars, icons, colors, sound effects, or TTS text files. The system will indicate to the student 
that there is a problem and will help the student to decide if he or she should repeat the task, change 
the speed, move to another mode or feature such as word building, or listen to the teacher example 
again. Default options will be established based upon the specific performance issue and will be 
determined by, for example, the VIV feature. 
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^ Kysj 6 4p^0C^^'r'5cfc 5 tc^f c 
ifc. ^'St,H3:CCDXr3TtS*B&3:^4fc»^^^L&l^C 

XT^Ai^cc^sn-S^W-WfS. 7-i>*aDrf# 



CIO) 
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[0 0 6 1 ] /^':7:^--'V>XtC^gT-g>>'^- K>^^* ^^^ 

[0 06 2] i.>^fc/o^^io^^i>^m(Dmmmmr^ 

^^yyt-^>:^^m'rWS>y'i-\'^^y^rj:^^^ 

[0 0 6 3] ^^:^^;I/^llf|iWb/cJ3. ^ft^Bt^ItU 

A:^y>e^2 f^iST^c 40 

rj::^^'oh(O^Lmmcik-<^Cti>. •ry:tJ\yh^O 50 



!f§P3 2001-159865 
18 

-s*fg;&y^-r^-1^ FSi±. »tRT-&ci:?^r#^„ 

5fe*O^^<^:^^i00^^%. 1f F • • 1f FJC^' 

1:?^^. ^^\miim^-^ti. ^^y :^--^iy7.tRm^ 

[0 0 6 4] ^^f^'^m^'P^ X,XJy A - Y^^v'P^ti 

C(Dm^^)\^y'<0-mtljX\t. "Try 
this,., in the word *gradu 
ate' the *d' is pronounce 
d witha j' sound as in *j 
ar' ( C:n«riS-5?^TT3 1^0 -^iS 'graduat 
e' iC^-l-tS 'd' 'jar' ^C*St:f^ 

O/c St) Ogt^P^DmigSlIU* f - - oab * 

[0 0 6 5 ] ^ ■:;^-i>Cii/;^7^ACCj:or^^X Ffta 



(11) 

15 

[0 0 6 6 ] ^^±<Dx,^-^cmT^mc^u^tixi.>^ 

tlTl^^^o "Try this,, r rx^ji y-^-i^mU. l^:^ 

'&^mmx^^j:^(,ctj:'^xi^^>. —mtox. 

F-r-Si. COffi^ii^^tP^- Foy X F. 30 
*balk', 'talk'. *5j:CX 'walk' i^m 

wr^o <:(om. ^^u. mm^(07€(D^:/u- h\ v 

[0 0 6 7 ] ^MX. m^^^^<D^- FCCT^Wr a 
[0 06 8] '^^<j:>WM\%. i^^:? ^ ^ -I' ^X F 

Lx. e?te%s6tc^< ci4). 

th. &i>i.^:s:■~^^^m1R'r^cti>x^^. u-^)\.i)S 
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[0 069 ] ^^0[:)/^':7:t-'^>XOX32T y 
i^^xm^UX^^o fcfcL. — ocr)|?fSHr^5f^MtCfcC^r 

itt. umt^^m^tSMf^c^<^-^xm^(Dj:'!?^c^ t}7.^ 
A>f t ^ n/c X 3 T »; > ^ - T^-ii/j^^'^ y :/ u - 3 > 
!j<>f >Fch Lr<D#lRxnTCc:S'^iir*^$n'So Fll 

OD^^^^CD --^ — X ^ > X :3 r ^ -S* C ^ tiBSJn ^ - V > 

feicffltien-So 2 6tc. ig^^^xcDSi^^c^mu-^ 

;K;c*tT'E>S*XnT0Dr--:/>'l/5j)^fflSU. ^^-^;^e 
n3t^^CD*ffl;0>61tjS^n'SiS^^^XCD^U>- F^ 

(icmm^ J: oWffi L . fife©^* a i:t:«ST ^ c <b ^ r # 

o 

[0 0 7 0 ] ^^CDWf^CDXnrti. 1f >^^>7- F-^ffir 

X3ir^f#^ca€>. ^^c^i^^(DSSP^3^(Dxnr^ 

[0 0 7 1 ] ^-^(DmBcm-r^iEmitmmu^y^y 

-oxm^mt^t. MiEji^»/^ 7 F3ti/c7*=^^X F 

[0 07 2 ] $)^-^^^tifiLmm<om\iov":fy- f* 
^'r^fcmc:^LUst^ct^x^^o crihv-¥ 

[0 0 7 3 ] f#6tiyt«^-Xv-r>xriT^;t:. 

^l(Dr--y;b5 2) rtOcEiiSn^^ r-- 

— 07^*x F4^f^s^fg:^jcDM^cC'S^*tc>(^ur«t,^^ 

[0 0 7 4] ^6tC. ^'^-7 ^:^;;l/:x-1f-r> 
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[0 0 7 5] ^ 9 ^^:^5C(7)^li^tf ^/cS6CD— 
SiSti. Xt-':':7'9 0 8ic4dl^-C. -ecDtlcDU lo 
iStRSn. X7^^:r:7'9 1 OCC*5i.^T3lJR$n/cU :/:=^> 

[0 07 6] SI 0&^. Ts^rvV'^^ A\C^\^^^^(0 

mm,\y^yy^mmt i>w^^-n'& \ oooiUTiDi^ 

i^^j&^H— >^ ^m^. XT- 1 0 0 6 CC:fe 

0 1 o^cfe^i^^T". f'-'r/'iV^I^^T^cir'. f#6tifc 

Xr3T:05fi6cDxnTiit$je$tl. Xf- :7* 1 0 1 2 6C:fe 

X7"-^:7*i 0 1 4Ccfctir. ^cc^^u 

[0 077 ] l|n(D^^°^^^:?:^^5*^$n/cit^^^. X 
f^^:7'l 0 1 Bfcfel^r. ^i3&5^ilC!)^N'5^^:7^^ 
-^5^^. X7" >:7'1 0 1 Sic^-i^r. ft^u«-^;i'7&^PJS5 30 
xf'^:':^'! 0 2 Oic^ai^r. 

/^'^^^:7^C>[^T^Xr3T^^^#^n2>o X7"^:'-:/10 2 
[ 0 0 7 8 ] H 1 Xr-^;/-7'9 0 8 CC:|(5t't5 U X 

xDSjRiQ 1 occ*>&^-5u ^'X^cDxnTy >^5^'^c>pi• 
■r€>xr3r y>i^<D^wii 1 0 o?:^-r« CO— fi^diu 

riT^l/^y X>t:t. t7JMl^^>'t'^^^tC*tT^^>C0T\ 9 40 
0%&SXr3T^Jf"O, *t7J^C. Xf--:?:?*! 1 0 2iC:fe5 

s*iS<^c>tt-r-scn?)Xnr;:>^6. ^^tc^fur. 
sn-s^ii'et*. #ig "fox" /co/c5 o%^c-g>x 
IS "fox" cc^^ur^^^c^T^x h^^tf^cC^bn'So 

OC. ^^Sfeco^il "fox" (D^^Z^iT^^tcWT^Xn 
Ts&^^fi&^n-S. i^0JC)Mr^S. XT'-:/:?*! 1 0 6iC:fc- 50 
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C^r. "f " i "x" CD^^^^^^^^f-SU^^rX^^^^S 

V x>(D/£:d6^rc. -^n^n. " f " t " X " 

1 1 10(C:foC^T. XT' ^7 -7" 1 1 0 2^C:|bni'^g^CDS: 

" j ump s" . Mazy*'. 4oJ:t)^ "dog" ) 
*fLr. Xf">:;:7'l 1 0 4-^1 1 0 8 i|S]— cDttf^:?^?5S 

msti-S. xt-^^:?"! 1 1 2i;c4c5iir. SiaS85 

ie:>sr*>n«. xr-:.:7'i 1 i4{c*5i.^r. ^3feic*tu 

tx-^o -X<^> Xf^-y 1 1 6tC*>l^r. U c/X>:^^X 
:3T^n. U*:7X>CD^ffi&C*t^^f'-:/;l':^H^Jl£$tl 

[0 07 9] ^mR^^^^^^:^'r2^^^x:f-nm^. ^< 

'&:$:^c*-::^< U-:;X>(Dffi^-li^;^)^fla;E$nT:u^-5o fc;^c 

9 4-14 ^i^wm^mKmu^'yi^^\'^m'^^c,^ 

OfgWcDSSi^i^g'C t S J: ^ tCX^ S n/h U X 

[0 08 0] mwrx<0^m^m (Directed 
learitiff experience) — S^^W&S^ 
XrA^t^ii. T^^X h3&>6^^tc^f^L. CCD^^^ 

[0 0 8 1 ] BM(Dm§^'^^^CXm< (Listen 
to any word)- -r+X 

r^VV-^^^W^cC ^(DX^'P^ -f :/x b"- :^;(DfflAW& 
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:lo J: 5 n fcffi;g(Di^W^ V ^ y f^' T 

t:\ a5S0H^:=^^-;^#ffi>&^^^*7*^;^ h (c i o s 

ed captioning text) ^KST-E) C 

[0 08 2] y ;^:::.>ii^':3>:7'y'^>>^3> (List 10 
ening comprehension) -f^^t^X h 

LO 0 B 3 ] U^ty'W>y*h (Example an 
d p r omp t ) -T'+Xh(?:)7"+Xh/^^^^M 

[0 084] mm. y^u>y*h. ^ (Ex amp 1 

e. prompt, record)— ^-'Xt^AJS, C ti 

-omc^mt}^'^^(oy'u>^y'^^^^^^>i^^. ^y^'o 

^-F. y\y-XUE^m.'%'th'^^(Jmtii% 30 

[0 0 8 5] ^fa. y'Vi^y"}-. S^t (Exam 

pie. prompt, record, playbac 

[0 08 6] S^-CSiR-r^Sffi® ( S e 1 f - s e 1 

acted reinforcement) —^^t^^ 40 

•^;l':r->Xr'AtcT^7'feX-r6<ii305-c*So CcDVX 
•rA&S. "Try this. . . (cn^lSUrT^ 

«H tel. « u «isa?> ^3 n ^ e^OOC) X ^ - tcs-^i ^ 
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[0 0 8 7 ] y'u^^y^ V . S^. M 

(Example, prompt, recor 

d. playback, compare, displa 
y r e s u 1 t s ) - >'X7='A<D^^tfe^i^.-^S^&Cffiffi 

4- - V > X CD*^3^ ^tD^ftg^^rffl-^-ii ^ C ch fcT ^ ^ . 

[0 0 8 8 ] S:i£X^;l/ (G r a mm a r skill 

s) -c^-FMSI^* (:7'niJ^^A) <Dii5PicJ:0. i§ 
i^t^^^r^-So ^3^53:. 7"^x 

fP^L/. ^^«:t^X7^ACc|g^3tPS:^$cTA:t!"r'5o f 

ffll^TmiiCDIt^CDJIIiJtca-^liTfS^^fl^ "Try th 

[0 0 8 9 ] e^Xr- AP^cD5ti^(0mt^^ffl^-^t>'tia C 

^&ct-?X^t. PL94- 1 42tc:?piteL/cfi 
i3&5"C*S. CCD->X7"ACr)SS&tifi^«. f-^^^XFiO^ 

[0 09 0] fg^(D^W'r'2>Ktc^i. WP^Wi^cCP^C^^a 
^aC)am^6CD^W?:yX::i>i:/(>!:)»StC«C^fc 
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[0 09 1] ±m(omn\it. m(ic:^^m<DmMtj:mmm * 

[06] :$:|§0J^cj;^75'accfcC^^ISW"r^itlicDi$ffl 
[07] :*:|fe5aje:J:^*ffi{c4stir-S^l^cDfe^— ocD* 

[0 8] :^mM(fCj;:^:f^m(iCi^if^n^m<o^mrj:m 

[0 9] *lfeeBfCci;^^^CC4>lt€>^^:6^^liT^illi 

[010] ^mM^cj:^ys'm^^^^^^M\^'^)\^(omm 

[011] :^mn(D^mc^^x::ir v>^'(ommi^ 
■r?^n0r^-5. 

1 0 MmmmmBmiy::^-7-A 30 

1 2 a>t:z^--^->Xr^J>. ^ 
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14 {/ay'U^ yV' 

16 Kt}'f^<-(:^ 

18 vY:$7Cl3f;> 

2 0 A;^y>^ 

2 2 f'-f 

2 4 l^^Xfc*-:^ 

3 0 x+X h/«^^fe (TTS) ^^^wl/ 
Stt^^iSIII (ASR) ^t:?^-;!/ 

(V I V) 



3 2 

3 2 3. 

3 6 MfJ-t>^^ 

3 8 
40 
34 
4^ 2 
44 

4 6 
48 

5 0 

6 0 
6 2 
6 4 
6 6 
6 8 
6 8a 
6 8b 

68 c w^y^d^y 

6 8d 

68 e W^-^-^l^ 
6 8 f M#J^<':^> 



U X > :7 r 



-7 J fc">^^^'-'y;i/ 
0 >f > F 



an F i g- 



16 



18 



20 



10 





-30 












■^38 
























^34 





56 



Me 



^40 



12 



— 14 



46 



[i][F] 

Ha 



44 



44 



fit*/ 



50 



52 



22 



24 



26 



Hi* 
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[03] F i g. 3 



m7] p I g. 



68a 



6a 



T 




ifi 



62 



7 



19 



/68c /ooa /w j'Qc 

O El 12 C_] 
i$Jl: 19^ i4cjk mW 



66d /68« 



G8i 



C x+:^htAAtS > — 302 



I 



I 



304 



306 



3Qfl 



T 



I 



•312 
-3U 




702- 



^3 



I 



I 



704 
706 



«»SH»rS ~| — 708 




4 ] F i g» t 



[05] F i g. 5" 



40S 

i. 



DRi: — 



4t6 



r— : 



402 




400 



1 



§-—410 



ff£«K»y^ftiT I 412 





422 



3 



424 



7~1 



420 




502- 

S04^ 
S06- 

510' 
512' 
5U- 



c 



3 



T 



I 



I 



I 



1 



] 



500 
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36] 



^^^^ F i g« ^ 




608^ 



806 
1 \ 



T 




S12 




SIS- 
S1B- 

622 



[YES 



I 



I 




900 



T 



1 



1 ^906 




m 



I 



824 



X808 



&26 



I 



I 



810 



B2B 



I 



I 



812 





[Si 0 ] F i i 



V-1002 



t 



I 



1004 
1006 



I 



1006 



1 



-^1010 



1012 



Itimrm^^^fi h ^lOH 

j-^10lfi 

"-lOIB 



t 



I 



I 



I 



^1022 
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[SI 1 ] F i i( 



MOD 



THE OmK BROWN FOX JUMPS OVER THE lAZY TAIL. 
^.□7f B5:i 85X &5X 5Q3( SOX &5)( 85% 50% 5C% 8^ 



FOX JUMPS LAZr DOG'S 



1102 



I 



T->^h: POX 

M% 50% 



mm : ^ 



-^1106 



I 



— 1108 



I 



J— 1112 



p 7^-:^^ 


1 


50 


40 


90 



:7P> ^^CQ^t- 



GO 9 B 



G 1 0 L 



5/14 
15/00 
19/04 
19/06 
13/00 
21/06 
15/00 
15/28 



F I 
GO 9 B 



G 1 0 L 



5/14 
15/00 
19/04 
19/06 

3/00 



R 
S 

5 5 1 E 
5 7 1 H 



C72)|%H^# 



y :^'^^B 07747 

T-^lf:^^^ffl 07974 -X.^ — — i^' 
/S-^^T^-^^ 225 



(72)#fe0^« 



T^y:^a^S 07728 ^jx-- 
159 
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T^^Jij^^m 07922 :=-:x—>>^—U 
•i/ 3^ — 150 



(72)|fe3a^ 



T^U:^a^lI 08904 J^;3L— 

^, >^%-r^>K -fe^^-" U""> 

T-^y^^^^H 07922 

^. j^-^VA 3^y- 

49 

T y ;^^^H 07076 A — I' — 
U - h 558 



